A Note on M. N. Katehakis' and Y.-R. Chen's Computation of the Gittins Index

In a recent paper Katehakis and Chen propose a sequence of linear programs for the computation of the Gittins indices. If there are N' projects and project c has K^ states, then 5)*-1 K,. linear prc^rams have to be solved. In this note it is shown that instead of the K,, linear programs for project c also one parametric linear program with the same dimensions can be solved. 1. Introduction. In a recent paper Katehakis and Chen (1984) propose a sequence of linear programs for the computation of the Gittins indices. In this note we show a computationally more favorable approach by using parametric linear programming. Wherever possible, Katehakis and Chen's notation is followed. Consider the following version of the multi-armed bandit problem. There are N projects and project t; is at each instant of time in one of the states of the set • Sp == {1,2,.. ., A;^). After observing the states of each project, one project must be selected to work on. If project v is selected at time t and the state of the project is state i, then a reward R^ii) is earned and p^ij\i) denotes the probability that the state of project V is state y at the next instant of time (the states of the unselected projects are unchanged). The problem is to find a rule for selecting the project such that the expected total a-discounted rewards for a discoimt factor a E [0,1) is maximized. Gittins and his co-workers have shown the existence of numbers M^ii), 1 < / < K^, \ < V < N, such that if at time point / project o is in state x^it), I < v < N, an optimal rule is to select project t;*, where